0 bookmark(s) - Sort by: Date ↓ / Title /
The article explores the architectural changes that enable DeepSeek's models to perform well with fewer resources, focusing on Multi-Head Latent Attention (MLA). It discusses the evolution of attention mechanisms, from Bahdanau to Transformer's Multi-Head Attention (MHA), and introduces Grouped-Query Attention (GQA) as a solution to MHA's memory inefficiencies. The article highlights DeepSeek's competitive performance despite lower reported training costs.
A comprehensive guide to Large Language Models by Damien Benveniste, covering various aspects from transformer architectures to deploying LLMs.
This tutorial demonstrates how to fine-tune the Llama-2 7B Chat model for Python code generation using QLoRA, gradient checkpointing, and SFTTrainer with the Alpaca-14k dataset.
Qwen2.5-VL, the latest vision-language model from Qwen, showcases enhanced image recognition, agentic behavior, video comprehension, document parsing, and more. It outperforms previous models in various benchmarks and tasks, offering improved efficiency and performance.
This article provides a comprehensive guide on the basics of BERT (Bidirectional Encoder Representations from Transformers) models. It covers the architecture, use cases, and practical implementations, helping readers understand how to leverage BERT for natural language processing tasks.
The article discusses the implications of DeepSeek's R1 model launch, highlighting five key lessons: the shift from pattern recognition to reasoning in AI models, the changing economics of AI, the coexistence of proprietary and open-source models, innovation driven by silicon scarcity, and the ongoing advantages of proprietary models despite DeepSeek's impact.
The article introduces a new approach to language modeling called test-time scaling, which enhances performance by utilizing additional compute resources during testing. The authors present a method involving a curated dataset and a technique called budget forcing to control compute usage, allowing models to double-check answers and improve reasoning. The approach is demonstrated with the Qwen2.5-32B-Instruct language model, showing significant improvements on competition math questions.
This repository provides an overview of resources for the paper 's1: Simple test-time scaling', which includes minimal recipes for test-time scaling and strong reasoning performance. It covers artifacts, structure, inference, training, evaluation, data, visuals, and citation details.
The article explores the DeepSeek-R1 models, focusing on how reinforcement learning (RL) is used to develop advanced reasoning capabilities in AI. It discusses the DeepSeek-R1-Zero model, which learns reasoning without supervised fine-tuning, and the DeepSeek-R1 model, which combines RL with a small amount of supervised data for improved performance. The article highlights the use of distillation to transfer reasoning patterns to smaller models and addresses challenges and future directions in RL for AI.
Hugging Face researchers developed an open-source AI research agent called 'Open Deep Research' in 24 hours, aiming to match OpenAI's Deep Research. The project demonstrates the potential of agent frameworks to enhance AI model capabilities, achieving 55.15% accuracy on the GAIA benchmark. The initiative highlights the rapid development and collaborative nature of open-source AI projects.
First / Previous / Next / Last
/ Page 1 of 0